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Abstract 

The purpose of this paper is that of applying cost-effectiveness 
analytic techniques to decisions on teacher reciultnent and retention. 
The data are derived fro© tha U.S. Office of Education's Survey of 
Equal Opportunity for the school year 1965-66. Evidence relating 
teacher characteristics to student achievement Is combined with data 
pn the costs of obtaining teachers with different characteristics. 
This evaluation suggests that recruiting and retaining teachers with 
higher verbal scores Is five to ten times as effective per dollar of 
teacher expenditure in raising achievement scores of students as the 
strategy of obtaining teachers with more experience. Separate estimates 
are made for black and for white 6th graders In schools of the 
metropolitan North. 
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COST-EFFECTIVENESS ANALYSIS AND EDUCA? xONAL 
POLICY—PROFUSION, CONFUSION, PBOMISE 

Henry Levin 



Introduction 

While the title of this paper Is tripartite, I will devote only small 
attention to the "profusion" and ''confusion*' and most of my efforts to the 
"promise/* Over the last two decades our society has experienced a 
revolution In decision-making processes. Among the new family of manage- 
ment tools, the most universally used techniques are probably those 
relating to cost-effectiveness analysis. Broadly speaking, these tools 
enable us to estimate the probable costs and benefits of alternative 
management strategies and subsequently to select those approaches which 
yield the best outcomes for any given cost. 

At the same time that cost-effectiveness techniques have been 

refined and applied, we have become increatiingly aware of the failures 

of the large-city schools. In particular, the educational systems of 

the cities have failed to effectively teach or significantly motivate 

2 

large numbers of disadvantaged youngsters. The recent governmental 



Assistant Professor, School of Education and Department of Economics, 
Stanford University. An earlier version of this paper was presented at the 
34th Annual Meeting of the Operations Research Society of America, 
Philadelphia, November 8, 1968. 

^A recent overview can be found in Thomas A. Goldman (ed.) Cost 
Effectiveness Analysis ; New Approaches in Declslon-IlaklnR (New York: 
Frederick A. Praeger, Inc., 1967). 

^These failures have been so well recognized that they are topics 
of the dally press. For some Insights see Christopher Jencks, "Is the 
Public School Obsolete?" The Public Interest (Winter 1966), pp. 18-28. 
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response to these failures has been to Increase spending for the schools 
In order to cotnpensate for dlsadvant^es In the backgrounds of their 
students. Indeed, the Eletaentary and Secondary Education Act of 1965 
alone has provided over $1 billion a year In additional school expendl- 
tures for students from low-'lncome families* 

Profusion 

Given these enormous Infusions of dollars, school districts, state 
governments, and the U.S. Office of Education have been Increasingly 
concerned abo6t how to get the most Impact out of the additional financial 
support. These governments have looked increasingly to cost-effectiveness 
analysts for the answers, and the response has been a profuse outpouring 
of cost-effectiveness studies. ^ Interestingly, each of these studies has 
examined the relationship between total costs and a hypothetical set of 
outcomes without examining the particular programs on which the money was 
spent. That is, the process by which education is produced has been 
ignored, and only a gross relation between dollar expenditures and outputs 
iias been surveyed. 

Confusion 

Yet the decision maker is faced with the problem of how to spend 
additional r«^sources in the most effective way possible • In doing this he 
is handicapped by some formidable obstacles. First, there is little 
imanimity on what schooling output io or on how to measure a multi- 
dimensional array of outcomes. Second, there is almost no theory which 



Some of the most extensive are: Thomas I. Rlbich, Education and 
Poverty (Washington, D.C., The Brookings Institution, 1968). Robert 
Spiegelman, et al., Cost-Benefit Model to Evaluate Educational Programs, 
"Progress Report," Stanford Research Institute (March 1967), and 
"A Benefit/Cost Model to Evaluate Educational Programs," Stanford 
Research Institute (January 1968). Clark C. Abt, et al^, "Design for an 
Elementary and Secondary Cost Effectiveness Model," Contract OEC 
1-6-001681-1681, Report on the Mathematical Design Phase for U.S. Office 
of Education (February 1967). Jacob J. Kaufman, et al., "An Analysis of 
the Comparative Costs and Benefits of Vocational Versus Academic Education 
in Secondary Schools," Contract OEG-1-6-000512-0817, Preliminary Report 
for the U.S. Office of Education (October 1967). 
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describes the relations between schooling Inputs, the educational process, 
and schooling outcomes. And third, there Is even a great deal of vague- 
ness on what should be considered as schooling Inputs. For example, it 
has been suggested that students contribute to the education of fellow 
students and that teachers' attitudes may be more Important than other 
teachers' characteristics. Finally, even student performance on standar- 
dized achievement scores Is so confounded by the student's own social 
class, his abilities, and his general environmental milieu, that it has 

proven very difficult to ncar.urc sc'iool effect. ; rarntely fron thoa 

4 

causc'l hy othax influences. 

The result of all this confusion is that additional expenditures for 
education have been spent in very traditional ways, most particularly on 
reductions in class size and the addition of remedial specialists. This 
very unimaginative route is taken despite the plethora of alternatives 
that are available: new instructional technologies, radically different 
curricula, and different types of teachers represent possibilities that 
have been scarcely considered while schools do more of what they've 
always done with reduced class sizes and a few additional specialists. 
Unfortunately, the cost-effectiveness studies undertaken thus far have 
done little to delineate the most effective strategies for any particular 
objective (e.g., raising reading scores). Indeed, one study has stated 
this shortcoming quite honestly: "A key part of this final analysis, which 
Is missing completely from this study, is the analysis of how differences 



The sparsity of knowledge in all of these areas is demonstrated 
In James S. Coleman, et al . , Equality of Educational Opportunity 
(Washington, D.C.: U.S. Department of Health, Education, and Welfare, 
1966), Chapter III; and Samuel S. Bowles and Henry M. Levin, 'The 
Determinants of Scholastic Achievement," The Journal of Hum«i Resources 
(Winter 1968), pp. 3-24. For a discussion of the problems in doing 
cost-effectiveness analysis in education, see Samuel S. Bowles, 
"Towards an Educational Production Function," A paper presented at the 
Conference on Research in Income and Wealth, Iniversity of Wisconsin 
(November 15, 1968; and Henr/ M. Levin, "Cost Effectiveness Evaluation 
of Instructional Technology: The Problems," A paper prepared for the 
Commission on Instruitional Technology (Washington, D.C.: November 
1968) . 
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In program Inputs can affect the direct measures of achievement.**^ 

Yet at the same time virtually all of the studies overwhelm the 
decision-maker with discussions of Markov properties, algebraic manipu- 
lations. Integrals, computer algorithms, flow diagrams, mock printouts, 
simulation *'flndlngs** and so on. Vfylle these concepts are elementary 
to the analyst, they are bewildering to the man who Is managing the 
schools. Worse yet, when he looks between the partial derivatives and 
slgmas, he sees no answers to his particular problems. In short, we 
as cost-effectiveness analysts have extolled our skills in solving the 
problems of the schools; we have agreed to help the decision-makers; 
we have taken their money; and in return we have bestowed upon them the 
Jargon of our trade. We have not been very helpful* 

Promise 

This pessimistic introduction brings us to a crucial juncture. Can 
any cost-effectiveness analysis be done at this time that might lead to 
more efficient production of schooling? If the answer were an unequivocal 
no, we would have to CALL EXIT and return the forum to more productive 
discourse. In my view, however, we should continue program execution, for 
there is already promise of findings that should be helpful to educational 
decision-makers. I wish to emphasize that what follows is Just a beginning, 
subject to qualification and subsequent modification. Nevertheless, I 
believe that it is a meaningful beginning. 

If one were to attempt to help the school decision-maker spend his 
money more efficiently, where would he start? An obvious place to begin 
would appear to be teacher recruitment, for teachers* salaries represent 
about 70 percent', of current operating expenditures for the elementary 
and secondary schools. Thus we might want to ask two questions: 

(1) Which teachers' characteristics show a relation to a 

goal that most of us would accept for the schools, i.e.. 



See Robert Spiegelman, et al . (1968), p. 54, 
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student performance on a standardized test of verbal 
achievement? 

(2) What does It cost the schools to obtain teachers with 
different characteristics? 
Given answers to these two questions, ve wish to ascertain whether we can 
obtain teachers with more effectiveness per dollar of expenditure. 

The first question might be answered If we were to estimate a 
production function of the form: 

(1) A - F(X,Y,Z^, Z^) 

Where A Is the achievement score for an Individual, X represents a vector 
of social class and background influences which affect achievement, Y 
represents a vector of non-teacher characteristics for the schools, and 
Zy represents a vector of teacher attributes. Ordinarily the 

assumption is made that F Is convex to the origin and continuous throughout 
Its domain (and that the first order partial derivatives are positive and 
the second order partlals are negative). 

Corresponding to question (2) would be a budget constraint 

(2) B » (P^Zj P2Z2 +...-!• Pj^Z^) 

which In this case would apply only to the teachers* costs, where Pj^, Pj 
denote the prices of teacher characteristics Z^, Z^^ respectively. Let 

us call this a teachers* quality budget constraint, since we are assuming 
that teacher-student ratios are constant, and that the question before us 
Is that of obtaining teachers of a better quality for a given teachers* « 
budget.^ 

While we are using this example only for Illustrative purposes, this 
approach does have the advantages of keep.ng the problem down to a 



The ellmlnaf-'.on of class size as a parameter of achievement Is based 
on the fact that no rigorous study has shown a consistent relation between 
class size and achievement within the ranges of class size under consider- 
ation* For evidence that even drastic reductions in class size and student/ 
teacher ratios show little effect on standardized achievement scores see 
David J. Fox, "Expansion of the More Effective School Program,'* Evaluation 
of New York City Title I Educational Projects 1966-67 (New York: Center 
for Urban Education, 1967), pp. 32-AA. 
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manageable — but still meaningful — size. 

Assume that ve wish to maximize (1) subject to constraint (2). The 
solution to this problem would require obtaining each type of teachers' 
quality until its additional contribution to achievement (3A/3Z^) 
relative to its price (P^) were equal for all (1 » 1, k). That 

is:^ 

(3) iklU^ = 3A/3Z2 - ... • ?V3Zj^ 



What if the school decision-maker has no knowledge of production 
relation (1) or the relative prices (P^) in (2)? This is certainly likely 
to be the case in the present instance where the knowledge gap is so great. 
Yet, assume that the decision-maker does indeed wish to maximize 
(1). Then, as cost-effectiveness analysts, we would like to give him 
information as to which teacher characteristics represent "best buys" in 
improving achievement scores within the confines of a limited budget. 

We do possess information supporting the view that for any given 
salary, schools can probably recruit teachers who are more effective than 
those whom they are presently obtaining. For example. Table 1 shows two 
production equations for Negro verbal achieve^nt at Grade 12. The specific 
data were taken from the U.S. Office of Education's Survey of Educational 
Opportunity, and these particular equations represent those derived from 
original specifications with over 20 variables. Both teachers' salaries 
and an incomplete — ^but logical— measure of teacher quality, teachers' 



The derivation of this solution is assumed to be familiar to the 
reader. Others may refer to Paul A. Samuelson, Foundations of Economic 
Analysis (Cambridge, Massachusetts, Harvard University Press, 1961). For 
a formal proof see H. Hancock, The Theory of Maxima and Minima (New York: 
Dover Press, 1960). 

^Regression coefficients that were not significantly different from 
zero at the 5 per cent level were eliminated from the specification. For 
further discussion of these results see Samuel S. Bowles and Henry M. Levin, 
"Equality of Educational Opportunity: More on Multlcollnearity and the 
Effectiveness of Schools," The Journal of Human Resources (Summer 1968), 
pp. 393-400. 
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verbal scores, show strong relations to student achievement. Tet, the 

teacher's verbal score seetns to represent a "better fit" in explaining 

variance in student achievement. This suggests that schools are not 

properly considering the price-productivlty relation for the different 

9 

characteristics of teachers obtainable at different salary levels. 

TABLE 1 

Regression liquations for Negro Verbal Achievement at Grade 12. 



« Negro student's verbal score 

* Reading material in home 

Xj • Siblings (positive equals few) 

X, " Parents' education 

Xj - Science-lab facilities 

X^ • Teacher salary (ill thousands of dollars) 
o 

X^ • Teacher verbal score 

(1) X^ - 33.40 + 1.99X2 ^ ^'^^^3 ^ ^'^^h ^ 0.062X5 + 1.78Xg 

(2.66) (4.34) (4.49) (3.18) (5.95) 
» .1506 
X'X » .68 • Measure of collinearlty 

(2) X^ = 19.49 + 2.09X2 ^ ^®^^3 ^ ^'^^h ^ °-050X5 + 1.24X7 

(2.82)^ (4.25) (4.38) (2.58) (7.14) 
R^ - .1633 
X'X » .68 • Measure of collinearity 

Note: (1) The t-values appear in parentheses; (2) defini- 
tions of the variables appear in James S. Coleman, £t al. , 
Equality of Educational Opportunity , U.S. Department of Health, 
Education, and Welfare (Washington, D.C.: 1966), Volume II, 
pp. iii-vii. 



An alternative explanation is that price-level differences among 
schools in a geographically dispersed sample create measurement errors in 
salaries, whereas no such problem arises with teachers' verbal scores. 
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Production Estimates 

What follows are the results from admittedly early representations 
of (1) and (2) which I believe yield insists into the teacher recruit-- 
ment problem. Eric Hanushek has estimated educational production func- 
tions for black and for white 6th graders in metropolitan schools. 
Using standardized achievement scores as measures of output and other 
data on inputs from the Survey of Equal Opportunity data, Hanushek 
estimated relations similar to (1) for whites in 471 elementary schools 
and for blacks in 242 elementary schools in the metropolitan North. Thus 
the analyses were cross-sectional single equation estimates for 1965-66 
done separately for black and for white students. While Hanushek properly 
specified these functions using social class and other variables as 
arguments, we will discuss only the estimated relationships between 
teacher characteristics and student verbal score. 

TABLE 2 

Means and Standard Deviations for 
Samples of Negro and White 6th Graders 



Negro White 

Mean St. Dev. Mean St. Dev. 

Student Verbal Score 26.68 4.20 35.70 4.54 

Teacher Verbal Score 23.98 1.80 24.77 1.43 

Teacher Experience (Years) 11.29 4.00 11*88 4.56 

Source: Eric Hanushek, 0£. cit . , p. 39 and p. 75. 



"Thfe Education of Negroes and Whites," unpublished doctoral 
dissertation (Department of Economics, Massachusetts Institute of 
Technology, 1968). 



TABLE 3 

Output In Student Verbal Score for Each Additional 
Unit of Teacher Verbal Score and Experience 



Additional Points of Student Verbal Score 
Negro White 



Each Additional Unit of 
Teacher Verbal Score 



.175 



.179 



Each Additional Year of 
Teacher Experience 



.108 



.060 



Source: Estimated from results on p. 37 and p. 73 In Eric Hanushek* 



In general, Hanushek found two teacher characteristics that were 
consistently related to the verbal scores of 6th graders. These two 
traits were the number of years of teacher experience and teacher's verbal 
score. The means and standard deviations for these variables are shown In 
Table 2 and the estimated p^offs to each characteristic are displayed In 



Thus for each additional point of teacher verbal score the Negro 
students showed an Increment of .175 points and the white students an 
Increment of .179 points in student verbal score. For each additional ,i 
year of teacher experience, the test scores of Negro students were about 
.108 points higher and the test scores of white students were about .060 
points higher. 



These estimated payoffs represent approximate slope coefficients 
for linear relationships between student's verbal score and the specific 
teacher characteristics, extracted from an equation in whiclh other rele- 
vant explanatory valuables were also included in the relationships. 
Teacher's degree level and other traits showed no statistically significant 
association with student achievement. 



op . clt . 



Table 3. 
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Teacher Costs 

The relative prices for teacher characteristics are taken from injr 

12 

estlmatM of earnings functions for teachers. In this work I estimated 
the relationship between teachers' salaries and teachers* chars'^terlstlcs. 
The estimates were derived for four metropolitan regions considered as 
labor markets » and the data were derived from the same source as that used 
by Hanushek. 

Table 4 shows the annual dollar return to teachers for specific 
characterl8t:lcs within an easterr metropolitan region. VRille tMs result 
represents a linear function for an aggregate sample of teachers, results 
are available for non-linear forms of the equation and by sex and race of 
teacher analyzed separately. For Illustrative purposes, however, this 
equation will suffice. 

Among this large sample of almost 3,000 teachers, about $24.00 of 
annual salary was associated with each additional point of teacher's 
verbal score; males were receiving about $A00 more than females; and each 
additional year of college training was worch almost $400 to a teacher. 
Teachers with non-academic majors were recelv. ag about $160 move than were 
their counterparts who majored In elementary education or academic subjects; 
gradua>tes of teacher colleges were receiving less than graduates of other 
Institutions. For eadi additional year of teaching experience, teachers 
were receiving about $79, and there were also higher returns to each 
successive certification level and to dissatisfaction with the racial 
composition of one's students ("discrepancy on proportion white"). 

What is of particular interest to us is that the aporoxlmate annual 
cost to the schools of obtaining a teacher with an additional year of 
experience was about $79 and that of obtaining a teacher with an additional 
point on the verbal scale was about $24, ceteris paribus . Applying these 
estimates to the results on Table 3, we can estimate the approximate costs 
of raising student test scores with two strategies: recruiting and 



10 

'TIenry M. Levin » "Recruiting Teachers for Large-City Schools," 
(Washington, D.C.: The Brookings Institution, forthcoming). 
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TABLE 4 

Estimation of Earnings Functions 
for Eastmet Teachers , 

Slope t 



Teacher Characteristics 


Coetziclent 


Dtaciscic 


Verbal Score 


$ 23.98 


5.6 


Female 


-398.59 


10.1 


Years of schooling 


396.04 


17.8 


Miscellaneous major 


159.73 


3.5 


Graduate of teachers' college 


-125.73 


3.0 


Years of experience 


78.91 


36.0 


Certification level 


564.09 


23.1 


Discrepancy on Proportion White 


18.27 


2.3 


Mean Salary 


7,084.56 




Standard Deviation 


1,679.76 




h 


.80 




R 


.65 




Sample Size 


2921 





retaining teachers with more experience, and recruiting and retaining 
teachers with higher verbal scores. 

Some Findings 

Applying these returns ve derive Table S« which shows the relative 

costs of improving student performances under alternative recruitment 
13 

strategies. It is important to emphasize the relative costs of each 



These costs were obtained by applying the teacher's experience and 
verbal score salary coefficients in Table 4 to the production coefficients 
In Table 3. It was assumed that the additional effort would have to be 
maintained for the first five years of schooling in order to obtain the 
sixth grade results shown in Table 3. Therefore « the present values in 
Table 5 represent additional expenditures for the previous fii^ years 
compounded at a 5 percent rate of interest and divided by an average class 
size of 30 in order to obtain a per-student figure. 
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strategy rather than the absolute ones* In teme of relative costs* ' 
for a given test score gain for Negroes, it appears that obtpj^nlng 
teachers with higher verbal scores is about 1/5 as co: tly a?; ottainln;; 
more tt»acher experience; and the teachers' verbal score routo is ten 
tines as efficient as teacher's experience per dollar of expenditure f or ^ 
increasing the verbal scores of white students « The obvious policy 
inplication is that school districts are obtaining too much experience 
as against verbal proficiency. Accordingly « the schools should try to 
Increase the recruitment and retention of verbally able teachers while 

paying somewhat less attention to experience. How much tradeoff should 

' 15 
be made is not evident given our linear results. 

TABLE 5 

Relative Costs of Increasing 
Student Verbal Achievement 



Approximate Cost for Increasing a 
Student's Verbal Score by One Point 

Strategy Negro White 

Teacher's Verbal Score $ 26 $ 26 

Teacher Experience $128 $253 



Another interesting observation is that teacher experienco appears 
to be twice as effective per dollar of expenditure for Negro students 
it does for white ones. Giving equal weights to point gains for whites 
and Negroes, the schools might wish to assign their more experienced 
teachers to the schools attended by Negro students for higher total 
yields. That is, the more experienced teachers should be redistributed 
to the Negro schools. 



The additional costs are probably biased downwards because the 
original salary data from which costs are estimated did not include 
fringe benefits. 

^^That is, our production estimates do not satisfy the conditions of 
the ficicoxid order partial derivative set out for equation (1) nbovo. 
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These findings are not the final answer by any means. They are 
neant to be illustrative rather than definitive. There are grounds 
for expecting specification biases on both the production and cost 
sides. Yet, it would take enormous biases— all in the sane direction— 
to offset our finding that it appears far more efficient to improve 
student achievement by raising teachers' verbal score than by increasing 
teacher experience. 

Further efforts in expanding this analysis and subjecting the present 
findings to additional scrutiny are being carried out by Hanushek of the 
U.S. Air Force Academy, Stephan Mlchelson and Samuel Bowles of Harvard, 
and the present author. We shall shortly see whether the promise that 
seens to be evident in this approach is justified. 
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